Test: add a repo-level release version consistency test - #297
Open
AmaadMartin wants to merge 1 commit into
Open
Conversation
This was referenced Jul 30, 2026
AmaadMartin
force-pushed
the
feat/release-version-consistency-test
branch
from
August 7, 2026 14:33
6cdf40d to
5770a21
Compare
release-please keeps core, dev, integrations and the root package in one linked-versions group, but nothing enforced that invariant: a partially applied release could leave the workspaces and .release-please-manifest.json on different versions with fully green CI. This test reads the four package.json files, the manifest and release-please-config.json at runtime and asserts they all declare one version, and that the set of packages it checks is exactly the linked-versions group, so adding a workspace to the release group without extending this test fails loudly.
AmaadMartin
force-pushed
the
feat/release-version-consistency-test
branch
from
August 7, 2026 15:11
5770a21 to
90d349b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A
Problem:
release-please-config.jsonputs the root package,core,devandintegrationsin onelinked-versionsgroup, so they must always publish the same version. Nothing enforced that. A release that bumpscorebut skipsdevkeeps every per-package test green, because each package only checks itself. Only a test that reads across packages catches the drift.Solution: One integration test reads the four
package.jsonfiles and.release-please-manifest.jsonat runtime and asserts they declare one version. It compares them as a single record, so the failure diff names every file that drifted. A second case asserts the packages it checks are exactly thelinked-versionsgroup, so a new workspace cannot join the release group and stay unchecked. No version literal appears in the test.This branch was 52 commits behind
main. I rebased it onto9360bf24and force-pushed. The diff is one new file.Collision check: I listed all 666 open PRs on the fork and inspected every version- or release-adjacent one. No other PR touches
tests/integration/release/. PRs #258, #417 and #619 add per-packagesrc/version.tstests undercore/test/anddev/test/. That is a different concern and a disjoint file set.Note on
JSON.parse: the parsed value is typedunknownand narrowed by a type guard. Typing it directly as the expected shape would be an unchecked assertion, which the repository guidelines forbid.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
tsc --noEmitdoes report pre-existing errors in other test files, for examplecore/test/a2a/agent_card_test.tsandcore/test/utils/live_connection_utils_test.ts. They are onmainalready andts:checkis not part of the validation workflow, so I left them alone.The file lands in the
integrationvitest project, whichnpm run test:coverageruns. That is the script the validation workflow calls, so the test gates CI.Proof each test can fail. I mutated the repository files one at a time and confirmed each mutation turns the suite red. I reverted every one with
git checkoutand re-ran the suite green.dev/package.jsonversion1.6.0->1.5.9- "dev/package.json": "1.6.0"/+ "dev/package.json": "1.5.9"core/package.jsonversion -> the number160+ "core/package.json": 160unreleasedexpected 'unreleased' to match /^\d+\.\d+\.\d+(?:-[\w.]+)?$/expected [ '.', 'core', 'dev', ...(2) ] to deeply equal [ '.', 'core', 'dev', 'integrations' ]packages.dev.component->dev-tools- "dev": ObjectContaining { "component": "devtools" }packages.dev-> the bare string"devtools"ObjectContainingdiffpackages-> an arrayExpected "packages" to be an object in release-please-config.jsonplugins-> an objectExpected "plugins" to be an array in release-please-config.jsonlinked-versionsplugin removedExpected a "linked-versions" plugin in release-please-config.jsoncomponents-> a stringExpected "components" to be an array in release-please-config.json[]Expected .release-please-manifest.json to hold a JSON objectThe third mutation isolates the semver guard: with all eight entries equal, the record comparison still passes and only the guard fails.
Manual End-to-End (E2E) Tests:
To watch it fail, change any
versionfield indev/package.jsonand re-run.Checklist